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Abstract. In statistical physics entropy is usually introduced as a global quantity 
which expresses the amount of information that would be needed to specify the 
microscopic configuration of a system. However, for lattice models with infinitely 
many possible configurations per lattice site it is also meaningful to introduce entropy 
as a local observable that describes the information content of a single lattice site. 
Likewise, the mutual information can be interpreted as a two-point correlation function. 
Studying a particular growth model we demonstrate that the mutual information 
exhibits scaling properties that are consistent with the established phcnomenological 
scaling picture. 
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1. Introduction 

In statistical mechanics the physical properties of fluctuating systems are usually 
described in terms of correlation functions which are defined as the expectation values 
of products of observables located at different points in space and/or time. For example, 
the critical equilibrium state of the Ising model is known to be characterized by spin-spin 
correlations of the form 

G(i,j) = (8 i S j )~\i-j\*-^. (1) 

Here Sj = ±1 is the local classical Ising spin at site i, (. . .) denotes the ensemble average, 
d is the dimension, and r] is the associated critical exponent. 

Another important pillar of statistical physics is the concept of entropy which 
describes the information content of a system. More specifically, if a randomly evolving 
system is characterized by a set of possible configurations (microstates) c£fi with a 
probability distribution P(c), the amount of information needed to specify a particular 
configuration c (in bit times In 2) is given by 



S{c) = - In P(c) 



(2) 
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Averaging over all configurations one obtains the Boltzmann-Gibbs or Shannon entropy 



which describes the mean information content of the system. For example, in the 
equilibrium state of the Ising model, where the probability of a configuration c = {sj} 
is given by the Boltzmann weight P(c) = Z~ 1 e~^ H ^ normalized by the partition sum 
Z, the average entropy is given by (S) = In Z + (3(E). Clearly, entropy as defined above 
is a quantity that characterizes the system globally. 

In the present work we suggest to look at entropy from a different perspective: 
Instead of defining entropy as a global quantity S, we want to use it as a local 
observable Si which describes the information content of a microscopic portion of the 
system, in the simplest case the information of a single site i. Such a local entropy is 
particularly interesting in models with infinitely many possible configurations per lattice 
site||| As a natural candidate, we will study here a particular growth process, where the 
height above a lattice site is unrestricted. 

The local entropy Si, as will be defined below, can be viewed as a special kind of 
one-point function. Likewise it is possible to study the joint entropy SV,- at two different 
lattice sites i and j. If these sites are uncorrelated one expects that Sij = Si + Sj. 
Therefore, it is useful to consider the "connected part" 



which is known as the mutual information in information theory [T]. Roughly speaking 
quantifies how much information site % has about the state of site j and vice 
versa. This concept can be easily generalized to n-point functions by considering the 
corresponding multivariate mutual information. Note that this concept differs from 
previous studies, where the mutual information between sections of a bipartite system 
was studied [2]. 

Entropic correlation functions like the mutual information differ from ordinary 
correlation functions insofar as the logarithm is a nonlinear function and therefore 
involves arbitrary high powers of the local field variables. For this reason it is not 
obvious whether such a correlation function exhibits the same type of phenomenological 
scaling laws as ordinary ones in the vicinity of a phase transition. However, the results 
of the present work suggest that it is possible to establish a set of consistent scaling 
laws. 

| If there were only a finite number of possible configurations per site the local entropy would be just 
a linear combination of ordinary local observables which is not expected to yield new insights. For 
example, in the case of Ising spins, where each site carries no more than one bit, the local entropy could 
take only two values and hence can be expressed in terms of the spin variables. 




(3) 



c 




(4) 
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Figure 1. Dynamical rules of the growth process, (a) Particles arc deposited 
everywhere at rate q. (b) Solitary particles and particles at the edges of plateaus 
evaporate at rate 1. (c) Particles from the middle of a plateau desorb at rate p. (d) 
Deposition and evaporation is forbidden if the resulting configuration would violate 
the RSOS constraint ©. 



2. Definition and properties of the growth process 

2.1. Definition 

As an example of a model with infinitely many possible configurations per lattice site, 
we study a simple solid-on-solid growth process which was discussed some time ago in 
the context of non-equilibrium wetting [Hill]- The model is defined on a one-dimensional 
periodic lattice with L sites labeled by i — 1, . . .,L. Each site carries an unbounded 
variable hi — 0, 1, . . . which describes the height of an interface above an inert substrate. 
Moreover, an effective interaction (surface tension) is introduced by imposing the so- 
called restricted solid-on-solid (RSOS) condition 

\h - h i±x \ < 1 , (5) 

i.e. neighboring sites must not differ by more than one unit. The model evolves by 
random-sequential deposition and evaporation constrained by this condition (see Fig.[TJ). 
It is controlled by two parameters, namely, a growth rate q, and another parameter p 
for the desorption from the middle of plateaus which allows one to interpolate between 
equilibrium and non-equilibrium (see Ref. jl] for further details). 



2.2. Scaling properties 

The growth model defined above is known to exhibit a continuous phase transition from 
a bound to a moving phase at a particular threshold q = q c {p)- This transition can 
be described in terms of two different order parameters, namely, the interface width w 
defined as the standard deviation of the height, and the density no of contact points at 
the bottom layer. Moreover, the critical behavior is characterized by a typical correlation 
length £j_ and correlation time £m. In the stationary bound phase close to the transition, 
where the critical parameter 



e = q c (p) ~ <? 



(6) 
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class 


V 


Qcip) 


a 


z 






C 


9 


(3 


bKPZ+ 


0.1 


0.61117(1) 


1/2 


3/2 


1 


3/2 


1/2 


1.184(10) 


1.776(15) 


bEW 


1 


1 


1/2 


2 


2/3 


4/3 


1/3 


3/4 


1 


bKPZ- 


2.0 


1.23237(1) 


1/2 


3/2 


1 


3/2 


1/2 


0.228(5) 


0.342(8) 



Table 1. Values of the critical parameters and the expected critical exponents [3] 
of the growth process in 1+1 dimensions. The exponents are related by the scaling 
relations 9 = f3/v\\, a = z = v \\l v ^ an d in 1+1 dimensions by V\\ = ( + 1. 



is small and positive, these quantities scale as 

w ~ e" c , n ~ e p , £ ± ~ e~ u± , f y ~ e - "" , (7) 

provided that the system size L is large enough. At the critical point q = q c 

(e = 0) one finds instead an asymptotic time dependence of the form 

w~t a/z , n ~t" e , U~t l /\ (8) 

where a = (/v±, = /3/v\\, and 1/z — u±/u». Starting with a flat initial state in the 
bound phase near the critical point one observes a crossover from (jSJ) to (J7j) which can 
be expressed by scaling forms with certain universal scaling functions. For example, one 
finds that the interface width grows with time according to the scaling form 

w(t,e) ~ t a W{te v \\). (9) 

The values of the critical exponents and the scaling functions are determined by the 
universality class of the phase transition. In the present model the parameter p allows 
one to choose between three different classes, namely, the bounded KPZ class with 
positive and negative nonlinearity (bKPZ±) for p > 1 and p < 1, as well as the bounded 
Edwards Wilkinson class (bEW) for p — 1. The expected values of the critical exponents 
and the corresponding critical thresholds are listed in Table [U 

2. 3. Exact solution for p = 1 

For p — 1 and q < 1 the model defined above is known to relax into an Boltzmann- 
distributed equilibrium state obeying detailed balance. This state is characterized by 
the partition sum 

z = E-EIR = E? E - lhl ( 10 ) 

hi h L i=l {h} 

which runs over all configurations compatible with the RSOS constraint (jSJ). To 
see this note that in a model without the RSOS constraint (jSJ) each site would 
independently perform a bounded biased random walk in height direction. These 
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decoupled random walks would evolve into a stationary state obeying detailed balance, 
where the probability of finding the value h is proportional to q h . Clearly, such a 
decoupled system would be described by the partition sum ( fit)]) with unrestricted 
summation. Then, imposing the additional constraint flSJ), it is easy to see that detailed 
balance is not violated and that the Boltzmann weights are preserved, - the only thing 
what changes is the summation (flOl) which is now restricted to configurations satisfying 
the constraint (jSJ). 

The stationary state for p = 1 can be described in terms of a transfer matrix 
formalism pEJ[6j[7] by reorganizing the partition sum ( 1101) as 

L 

z= j2---Y,n < i iht+ht+i)/2 ( n ) 

hi hi, i=l 

= ^2 Q (hl+h2)/2 ^ Q {h2+hs)/2 X] q (hL+hl)/2 

hi ha h L 

hi h% 

(12) 

where T is the transfer matrix with the infinite-dimensional tridiagonal representation 



Y,Th lM = Tr[T i 

hi 



T 



( 1 q 

1/2 



q 



q 



1/2 

q 

3/2 



3/2 



g 2 g 5 / 2 



V 



\ 



/ 



(13) 



Defining canonical basis vectors \h) and (h\ the probability of finding site i at height h 
is then given by 



Tr 



P{h) 



T^ihlT 1 



(h\T L \h) 



(14) 



Tr[T L ] Z 
The transfer matrix T is symmetric and has a non-degenerate spectral decomposition 



(15) 



n=0 



Since 



with real eigenvalues and pairwise orthonormal eigenvectors \4>^) and (<f>^\ 
a high power of such a matrix is dominated by its largest eigenvalue, we may therefore 
approximate T L in the thermodynamic limit L — > oo by 



(16) 



where X q = Xq denotes the largest eigenvalue of T with the corresponding eigenvectors 
(4> q \ and \4> q ). Consequently Z w so that the expectation value of finding a site at 
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height h is given by 

P{h) = (h\j> q )(j> q \h) = \(h\j> q )\\ (17) 

Remarkably, the transfer formalism reminds one of the Dirac formalism in quantum 
mechanics although the present problem is classical. 

For general q < 1 the determination of the dominating eigenvector is non-trivial and 
to our knowledge a closed solution is not yet known. However, close to the transition, 
where e = 1 — q is small, the eigenvector <p q {h) can be approximated by an Airy function 
of the form [4] 

(18) 

f™Ai 2 (z)dz « 0.491697 is 



<f>(h) 



.4 



Ai 



(3e) 1/3 /i + 



where Zq ~ —2.33811 is the largest root of Ai(z) and A 
the corresponding normalization. 



3. One-point function: Local entropy 

In a growth process the local entropy at site % is given by 

Si = -J2 p (hi)^P(hi). (19) 

hi 

Near criticality, where e = q c — q is small, the probability P(hj) = P{h) to find the 
interface at height h is expected to obey the scaling form 

P(h) ~ e c $(/ie c ), (20) 

where $(z) is a scaling function determined by the universality class selected by p. 
Replacing the sum in fTl~9|) by an integral and inserting this scaling form one can show 
that the one-point entropy scales as 

^-C-Chie (21) 

in the limit e — > 0, where 



oo 



C = - / $(^)ln$(z)dz (22) 



o 



Since the universal scaling function is only defined up to a rescaling the constant C 
is non-universal. For p = 1, where the exact solution (fl8|) leads to the scaling function 
$(z) = 3 1 / 3 A- 1 Ai 2 (3 1 / 3 z + zq), we obtain the numerical value C ~ 0.650832. 

To confirm the predicted scaling behavior, we measured the local entropy in a 
numerical simulation (see Fig. [2]). In the Edwards- Wilkinson case p — 1 the numerical 
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Figure 2. Local entropy as a one-point function in the stationary state (see text). 



data (red dots) agree very well with the transfer matrix results. Moreover, the 
logarithmic decay with a slope —0.34 is in agreement with the expected exponent 
( = 1/3. For p = 0.1 the measured slope —0.43 is not in full agreement with the 
expected exponent £ = 1/2 of the bKPZ- class. This confirms that the crossover from 
EW to KPZ behavior of this particular model is very slow, see Ref. [5] for a detailed 
discussion. 

4. Two-point function: Mutual information 

By means of the RSOS constraint (jSJ) the lattice sites are not independent ,but 
exchange some information about their local state. As outlined in the introduction, 
this information exchange is most naturally quantified by the mutual information 
= Si + Sj — Sij between two sites i and j. Because of periodic boundary conditions 
the mutual information will only depend on the distance r = \i — j\ between the points, 
i.e., I(iJ) = I(r). 

In the special case of p = 1 the transfer matrix formalism provides a tool to calculate 
the mutual information analytically. To this end one has to compute the joint entropy 




(23) 



in terms of the joint probability P(hi, hj). This probability is given by 




(24) 



{h}ij 



were the sum runs over all possible configurations while keeping the heights hi and hj 
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at the positions i and j fixed. Using the transfer matrix method these probabilities can 
be expressed as 

Pftiy= «mw, (25) 

Xq 

Although it is not trivial to calculate the mutual information from this expression, one 
can find useful approximations in the limit of short as well as very large distances. 

Short distance limit: 

If the distance between the two points is much shorter than the correlation length, one 
can estimate the decay of the mutual information as follows. First note that the mutual 
information can be written as 

= Si-Siv (26) 
with the conditional entropy 

% = -J2 P ( h ^12 P ( h i\ h J lnP ( h i\ hi )- ( 2? ) 

hi hj 

For neighboring sites the conditional probability to find sites i + 1 at height hi+i given 
that site % is at height hi reads 

{ h i+ h i + l 
W if l".-"«l<l (28) 
otherwise. 

In the limit q — > 1, where (h i+1 \(j) q ) ~ (hi\4> q ), this expression reduces to 

1/3 if [hi-hi+il <1 (2Q) 
otherwise. 

This shows that on short distances the interface height hi increases or decreases by one 
unit or stays at the same height with equal probability as we move to the neighboring 
lattice site. In other words, on short distances the interface describes an unbiased 
random walk in height direction. Therefore, if the distance r = \i — j\ is sufficiently 
larger than 1, but still smaller than the correlation length, the central limit theorem 
implies that the conditional probability P(hj\hj) is approximately given by a normal 
distribution centered around hi with the width proportional to \fr. Consequently the 
conditional entropy is of the form 

Hihilhj) ~ # + ^ln(r) (30) 

with the numerical offset H = 1.216206. Inserted into ( |26|) and using (1211) this leads to 

7(r) ~ / -^ln(e)-^ln(r) (31) 
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with the numerical value Iq = —0.565374. 
Long distance limit: 

In the limit where r = \i — j\ is much larger than the correlation length, the two sites 
are almost statistically independent so that the joint probability distribution P(hi,hj) 
differs only slightly from P(hi)P(hj), i.e. 

P(hi,hj) = P(h i )P(h j ) + r lhtihj , (32) 

where t] h . h . <C 1. This allows the mutual information to be expanded as 
I(i,j) = S(h i ) + S(h j )-S(h i ,h j ) 

= -J2 P(hi)P{hj) \xx[PQii)P{hj)] + P {K hj) In P(h u hj) (33) 



hi,hj 



= E K 1 + wpik,))^, + + ° ( " 3) ■ 

To compute the small deviation r)(hi, hj), we insert the spectral decomposition (I15p into 
Eq. (1231) one obtains 

P^k,) = + f ^I^X^I^X^y^'feX^I^ . (34) 

S ^ ' n=l \ 

=P{h l )P{h j ) 

Thus we can identify 77/^. with the first summand, i.e. 

r) hi>hj = (^I^X^I^^^I^X^I^A-I^'I (35) 

where A = Xq/Xq is the gap ratio between the leading and the next-to-leading 
eigenvalue. Inserting this expression back into the expansion ( 1331) one can show by 
using the orthogonality of the eigenvectors (4> q \4>^) = that the first-order contribution 
vanishes. Therefore, to second order in rjh. ^. the mutual information is given by 

I(r) - \ ^(/^m 1 ^) 2 A"" = \e~ M . (36) 

hi,h h 



This means that in the long-distance limit the mutual information decays exponentially 
as I(r) ~ e~ r ^ x with the correlation length 

^=2EA- (37) 

The correlation length, which is determined by the first gap ratio A of the transfer 
matrix, depends on e = q c — q. As expected, one finds numerically that £j_ ~ e~ u± in 
agreement with the scaling behavior of Eq. (I18|) . 
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Figure 3. Mutual information I(r) plotted as a function of the scale-invariant 
combination re v± , leading to a data collapse according to the scaling form ([55)1 . Left: 
Numerical data for the bounded Edward- Wilkinson case p = 1 together with the short- 
and long-time approximations in Eqs. (|31[) and (|37[) shown as dashed lines. Right: 
Corresponding data collapse in the bKPZ case p = 0.1 (see text). 



Scaling form: 

The two asymptotic formulas (J3TT) and (1371) . which both depend on the scale-invariant 
ratio r/£j_, suggest that the crossover from one behavior to the other is given by a scaling 
law of the form 

I(r) ~ F(r/Cx) , (38) 



where F is a scaling function which is expected to be universal. Because of (131]) and 
( |37l) this scaling function behaves asymptotically as 

F< \ / J o-| m ^ forl«r«(i 
UJ ~ \ ±exp(-z/A) for r»£ ± ' 1 ' 

Note that in contrast to conventional scaling forms, there is no leading power law in 
front of the scaling function F in Eq. ( 1381 . 

To test this hypothesis we measured the mutual information as a function of r in a 
numerical simulation for various values of e. As shown Fig. [3] one obtains a convincing 
data collapse in the case p — 1. Similar results are obtained by using the transfer matrix 
formalism. For p = 0.1 the best possible data collapse is obtained for u± ~ 0.9 which 
differs from the expected value u± = 1 for the bKPZ- class. This discrepancy is again 
caused by the slow crossover from EW to KPZ in this model. 
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5. Conclusions 

In stochastic lattice models the local entropy describes the information content or 
uncertainty of the local state of a single lattice site. Likewise the mutual information 
describes how strongly two sites are correlated. These quantities are particularly 
interesting in models with infinitely many states per site, where they cannot expressed 
as finite linear combinations of ordinary correlation functions. This leads to the question 
how entropic observables behave in systems with a continuous phase transition. 

As an example, we have studied a simple growth model of a one-dimensional 
interface. In this model the interface at site i is described by a local height hi = 
0,1,2,... and thus it has infinitely many states per site. Moreover, the model 
exhibits an unbinding transition from the substrate controlled by the growth rate. 
Another parameter allows one to select various universality classes with different critical 
exponents. 

As for the local entropy, interpreted here as a one-point function, we find a 
logarithmic scaling behavior of the form 



where e = q c — q parametrizes the distance from criticality and £ is one of the critical 
exponents listed in Table [Q This result is expected since this exponent characterizes 
the width of the interface close to the transition. 

The scaling behavior of the mutual information between two lattice sites, 
interpreted here as a two-point function, depends on the distance r between the two 
points. For p = 1 we find the asymptotic behaviors 



where £j_ ~ e u± denotes the correlation length. We expect these limits to remain valid 
in the KPZ case p ^ 1, using the corresponding KPZ exponents. 

This asymptotic limits in Eq. ( HIT) suggest the general scaling form 



In the present model this scaling form can be confirmed numerically, leading us to the 
conjecture that the scaling function F is universal in the same sense as for ordinary 
correlation functions. However, in contrast to ordinary scaling functions, which usually 
describe the crossover between different power laws or the crossover from a power law 
to an exponential decay towards a constant, the function F describes a crossover from 
a logarithmic to an exponential decay. 

Moreover, it is important to note that there is no leading power law in front of F, 
meaning that the mutual information does not carry an intrinsic scaling dimension. 



St ~ C-Clne, 



(40) 




(41) 




(42) 
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Ordinary correlation functions carry an intrinsic scaling dimension which is usually 
determined by the scaling dimensions of the local observables. In entropic correlation 
functions, however, the logarithm involves arbitrary powers of local observables and 
therefore it is plausible that it cannot carry an intrinsic dimension. Whether or not this 
is a general feature of entropic correlation functions remains to be seen. 

The proposed concept of entropic one- and two-point functions can easily be 
generalized to n > 2 points by considering the so-called multivariate information 
between these points. Moreover, it is straight forward to apply similar ideas to quantum 
systems by replacing the local Shannon with the corresponding von-Neumann entropy. 
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